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together the contents of physics, mathematics, and computer science courses 
in a single course in which students worked on real-life problems. In order 
to assess academic achievement, three data sources were used: pretest and 
post test scores on physics tests for two samples of students, grade point 
average of all students enrolled in the curriculum, and students' grades in 
advanced engineering courses in comparison with the grades of students in the 
traditional courses. A second source of data came from studies that evaluated 
the acquisition of professional skills. Test scores on critical thinking 
skills, as well as self-evaluation and co-evaluation tests were used. The 
final set of data attempted to assess how the PBL curriculum affected 
students' attitudes toward mathematics, physics, computer science, and 
education in general. The PBL curriculum seemed to improve the academic 
achievements of these students, but student attitudes were very similar to 
those of students in the traditional courses in spite of their enhanced 
academic achievement. Additional studies are planned to investigate students 
attitudes toward the PBL course and continued academic achievement. (Contains 
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Effects of a Problem-based Learning Program on Engineering Students’ Academic 
Achievements, Skills Development and Attitudes in a Mexican University 1 

Rodrigo Polanco 2 , Patricia Calderon 3 , Francisco Delgado 4 
Instituto Tecnologico y de Estudios Superiores de Monterrey 

Abstract 

The aim of this paper is to report on a three-year follow-up evaluation of an 
experimental PBL integrated curriculum directed to students of the first two 
years of engineering. The PBL curriculum brought together the contents of 
physics, math and computer sciences courses in a single course. Instead of the 
students having to enroll in physics, math and computing courses, they had the 
opportunity to enroll in a single course in which they learned the contents of 
those subjects by solving real life engineering problems. In order to assess 
academic achievement, three data sources were taken: pretest- postest scores on 
Physics tests, grade point average, and students’ grades in advanced 
engineering courses. A second source of data came from studies that evaluated 
the acquisition of professional skills. Test scores on critical thinking skills, as 
well as self-evaluation and co-evaluation tests were used. The final set of data 
attempted to assess how the PBL curriculum affected student’s attitudes toward 
Mathematics, Physics, Computer Sciences, and education in general. 



Everything coincides, is the paranoid’s formula. However, as any psychotic formula, it 
conveys a certain amount of truth. Historians of ideas might, in a better way, provide a 
description of the turnabout of man. Many of the changes we have coursed as gender 
coincidently arise from different ambits of human reality. Thus, while present-day Physics 
runs into the Heisenberg’s principle, and Mathematics encounters Goedel’s theorem 
(Hofstadter, 1980), epistemology questions the immobility of scientific knowledge, the 
great economic models -and of other types- manifest their inconsistency. The proximity to 
other cultures questions many of our conceptions about the being human, (such as gender 
differences). Everything coincides. Uncertainty comes to sojourn in our house. Certainty of 
control abandons us (Anderson, 1990). 

Our certainties and longings for absolute control, of reality and of ourselves, have proven 
ineffective. Perhaps uncertainty was always in the core of humanity. However, the great 
narratives among which we lived (our conception of science, about accessibility to truth, 
the firmness we granted to symbolic systems, and above all, the mathematical system), as 
well as confidence in our small and daily tales about ourselves and about life, had permitted 
us to elude their persistent encounters. 



1 Paper presented at the 82nd Annual Meeting of the American Educational Research Association. Seattle, 
April 10-14, 2001 

2 Direccion de Efectividad Institucional, Rectoria Zona Sur. E-mail: rpolanco@campus.rzs.itesm.mx 

3 Direccion de Efectividad Institucional, Rectoria Zona Sur. E-mail: pcaldero@campus.rzs.itesm.mx 

4 Departamento de Matematicas, Campus Estado de Mexico. E-mail: fdelgado@campus.cem.itesm.mx 




3 



Effects of a PBL program on engineering students’ achievement, skills and attitudes 



2 



1 



These changes produce a new reality which we must now struggle with. Living in 
uncertainty with an education of certainties is a difficult challenge. 

Traditional teaching obtains its support from a rationalism that has proven its 
ineffectiveness in the present complex world where we live. Straight-line cause-effect 
ratios, our possibility of accessing truth, the pertinence of general models, all what has 
helped us reach to the point we now are, to develop science, to achieve a certain control 
over some phenomena, to expand our possibilities in an extraordinary fashion, today seems 
ineffective to respond to quite a more complex reality (Dreyfuss, 1993), with a demand for 
adequate answers to each case, with a greater number of interrelations, a communication 
much more within reach, and consequently, a reality unfolding at a faster pace. 

With all this the human being of today cannot continue to function from rationalism, he has 
outgrown his former birth cradle. The ineffectiveness of traditional education is also a 
product of the scopes of rationalism. It is their greatest achievement, and their limit. 

Most of all, because from there - from rationalism -, uncertainty is but a threat, something 
undesirable and hard to bear. Without modifying this discourse, we will find it very 
complicated to face dailiness. We cannot continue educating for the sake of control and 
certainty. Education must prepare adolescents to become adults who permanently live in an 
uncertain and changing world. A world of constant renovation, that also demands of them a 
permanent reinvention of their identity. 

How can that education be? An education in favor of flexibility, that will develop the 
capacity of expanding one’s own possibilities of alliance with others, where the only 
certainty rests in the conviction that nothing guarantees the success of our undertaking, but 
as far as we are able to see, the best option has been chosen. This education might, for 
example, give the adolescent another discourse about uncertainty. Propose that he confront 
uncertainty with the meaning we grant to it in the game, such as that emotion without 
which there is no entertainment. 

Other changes necessary in this education, more than an accumulation of information — as a 
professor strived for in the past - are meant to offer him certain coordinates that permit him 
to move about adequately in the sea of information he now has access to. In addition, 
beyond information, what the future professional requires is the capacity to act in a wide 
ambit of knowledge. The segmentation given to knowledge by the academy leaves it less 
functional for its application in real situations. It is quite often that a profession is learned 
when beginning to exercise it. This leaves the period of professional formation of little use 
for its purposes. 

This is the matter that led the academics of Me Master in Canada (Barrows and Tamblyn, 
1980; Barrows, 1986) to try to modify the pedagogy of teaching Medicine, starting from 
the knowledge history has revealed to them: that a student really does not learn until he is 
in the midst of the practical problems of his discipline. The proposal by this Canadian 
university has now been adopted by other universities in various countries (De Graaff, and 
Bouhuijs, 1993), and in various areas of Medicine (Maxwell and Wilkerson, 1990; Sobral, 
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1995; Vernon and Osokawa, 1996), Nursing (Williams, 1995), Odontology (Pereira, 1998; 
Aldred and Aldred, 1998), Psychology and Occupational Therapy (Reynolds, 1997), 
Educational Administration (Cordeiro, 1997; Copland, 2000), Engineering (Stevens and 
Wilkins, 1993; Woods, 1996), as also in levels of Basic Education (Achilles and Hoover, 
1996; Williams Hemstreet, Liv and Smith, 1998), and Media Superior (Glasgow, 1997; 
Jones, Rasmussen, and Moffitt, 1997), Dods, 1997; Milbury, 1997). 

Although the original proposal has its origin in the pedagogic and practical exposures 
referred to in the above paragraph, its repercussions reach beyond them, they touch the 
history of knowledge. Human understanding developed in this manner. This does not mean 
that the student be subdued to finding solutions, that scientists of the discipline referred to 
have solved. It simply points toward the fact of organizing the structure of the student’s 
knowledge in respect to problems, that allow him to find a significance to the already 
existing scientific explanations. To find the feasibility of a technique in order to solve a 
problem and also, a much clearer relation between theory and practice, to face the 
necessary interdisciplinary method (Morin, 1977) demanded by any solution of a real 
problem. 

Thus, knowledge comes as an answer to those questions that arise in the student from the 
case or practical situation, and not an answer — as it was in the past - to the teacher s 
questions. By its connection with other disciplines, as well as the cognitive motivation 
involved in solving mysteries themselves, this learning is more lasting and widens the 
student’s practical capacity (Inhelder, 1975). In addition, the context within which 
problems are solved forces the student to solve them in a group, and in this form, if work is 
adequately guided by the facilitator, the means also becomes a space to develop skills in 
communication, listening, negotiation, collaboration; in building alliances with others, all 
of which could be comprised in one term, team work. 

These three pillars of problem-based learning, interdisciplinary method and team work, in 
the summer of 1997 gave origin to what was called the “Principia Project”, a curricular 
program consolidated on Problem-based Learning, directed to students of the first two 
years of Engineering. In this project the students learn the contents of the basic subjects 
(Physics, Mathematics and Computing) by applying Problem-based Learning (Barrows, 
and Tamblyn, 1980). The aim of this document is to report results that show the advance 
achieved by the project in these four years of its existence. The following work section 
provides a detailed description of the program, and further there is a report of the 
evaluations it received. 

Antecedents and contextual definition of the proposal 

A trend observed in recent years, is to change the teaching-learning model of ITESM, a 
private institution of Mexican Higher Education recognized for its high standards in the 
formation of students. Without overlooking the area of knowledge, the new educational 
model emphasizes on developing various skills, attitudes and values (SAV) in students, 
such as teamwork, learning on one’s own, the use of technology, capacity of analysis, 
synthesis and evaluation, capacity to identify and solve problems, good oral and written 
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communication, and others. These SAV’s pretend to prepare the student so that he face the 
future with leadership and participation. 

During the last five years, various projects and pilot courses have been instrumented in the 
area of Engineering, which seek to emphasize on the development of SAV’s in the courses 
that are imparted. As a result of these projects, it was possible to observe problems that 
exist in teaching and in learning mathematics among our professors and students. For 
example, we found there is a low retention of knowledge in students; that the courses are 
exaggeratedly directed and centered in algebra; that there is an abuse in the use of orthodox 
rules and algorithms, and a complete development of a mathematical reasoning is not 
achieved. On the other hand, the courses were found to lack applications in the areas that 
interest students, and that the density of the programs restricts the use of applications. 
These conditions result in a very low motivation among the students toward mathematics 
and, in general, toward sciences, as their contents lack significance. 

The Principia Project 

The Principia Project was bom as an answer to the above situation, with the idea of 
surpassing the difficulties mentioned, and to help students in Engineering develop a 
mathematical and scientific culture that permits them to face various situations where 
successful proposals for physical and mathematical problems could be required. For such 
purpose, with Problem-based Learning (PBL) as reference framework, the mathematics, 
physics and computing program for the first two years of the career was redesigned. Instead 
of the students having to enroll in the traditional Physics, Mathematics and Computing 
subjects, they had the option of enrolling voluntarily in the Principia Project where, in an 
integrated subject the student acquired knowledge in those subjects by solving engineering 
problems. 




From the beginning of the academic semester the students are assigned to permanent teams 
where they will remain throughout the entire period. PBL develops under a system of 
collaboration, mainly through three types of activities: 

a) Solution of exercises. In this activity the students temporarily leave their basic teams 
and form transitory teams to solve small-scale problems, where the purpose is to 
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develop elemental competence in each area. The activity concludes when the students 
return to their basic teams to transmit, share and enrich the knowledge among the 
other members. 

b) Solution of complex problems. In this activity the students work in their permanent 
groups and confront large-scale problems, where the solution of exercises forms part 
of the solution of a more complex problem, requiring the integration of knowledge in 
various disciplines, as well as the use of technology for its development. 

c) Development of projects. An activity also developed in permanent teams, consists in 
the open solution of a complex situation. This situation implies acquiring additional 
knowledge to that contemplated in the curriculum, where future learning in major 
fields and in sharp problems in these fields is normally exploited. 

The elements normally considered when designing the problems are schematized in the 

following table: 

Dimensions of the design 
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Environment (E): refers to real situations that can occur when developing the 
activity in respect to the level of comprehension reached or used by the student, such 
as in the classic scheme. 

Curriculum (C): the contents on which the activity is centered and for which it is 
basically created. The curriculum is the traditional nucleus of learning; however, in a 
problem-solving activity it must be subordinated to other elements. In addition, in the 
Principia Project, the curriculum of every area must be considered. 

Analysis point of reference (AV): also refers to the curriculum of the areas 
integrated, but implying that previous (retrospective) purposes and future 
(prospective) objectives are retaken to enrich and enable the problem, foment a long 
term retention of other fields of knowledge, and detect the need for future knowledge 
Use of technology (T): the technological elements (software, laboratory, means, etc.) 
which the activity integrates. This dimension must establish an analysis of its 
plausibility, its significance, and the role it plays in that activity. 

Development of formative objectives (F): within the context of ITESM, this 
dimension takes care of the SAV’s mentioned in the Mission of this Institution. 



During a typical and complete problems session within the program, three stages are 
distinguished and are schematized in the table on next page. 



The complete stages of a problem-solving activity 



1 lie CUllipiClV v» - f' D 
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STAGE I: acquisition of 
knowledge 


STAGE II: collaborative 
learning 


STAGE III: problem 
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the time and guides the 
team toward the objective. 
Student: cannot interact 
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every member to 
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Elements of 
action 


• Areas of specialization 
are defined. 

• The expert teams are 
formed based on each 
student’s skills. 


• An activity of immediate 
application is defined that 
will allow interaction and 
exchange of experiences of 
each expert with his team- 
mates. 


• An applied problem is 
proposed that involves 
the application of 
previous stages, 

• And of other contents 
within the framework of 
the analysis proposed. 


Work form 


Every basic team is divided 
to form expert teams 
integrated by elements 
from different teams. 


The basic team meets to 
solve an intermediate activity 
where every expert provides 
his knowledge to the group. 


The basic team guides 
itself as a team to solve the 
problem. 


Evaluation 


Every expert team exposes 
and is evaluated for its 
fulfillment of the specific 
activities listed in the 
script. 


This is field evaluation (as to 
efficiency and work form of 
the team) and mainly in the 
scope of the objective 
proposed in the activity. 


The evaluation is centered 
on the in situ report 
prepared by the team and 
the annexed report that the 
team may hand in the next 
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The idea of these stages is to gradually introduce the student to a subject until he can end up 
with an applied problem that has its roots in the control of such problem. However, stages I 
and 11 may be absent in some activities, and will invariably be absent in any integrating 
evaluation activity. However, their design is the same, although evidently, the script 
changes. The fundamental requirement in those “incomplete activities is that other of the 
activities has previously covered these elements. 

The creation of a problems network under these considerations, where a framework of 
analysis is established, permits to evaluate the recurrence to previous subjects and the 
perspective toward future subjects. With this, the entire network results more important 
than the problem, as it allows to consistently giving a meaning to that activity within the 
course. 

The following work section presents some of the results obtained to date about evaluation 
of the program in three areas: academic achievements, development of skills, and attitudes 
of students. 

Academic achievement 

In order to assess academic achievement, three studies were carried out. Every study will be 
reported separately. 

Math and Physics Concepts 

In the first study, a pretest on Physics and Calculus concepts was taken to two samples of 
students. The first sample was composed y the students enrolled in the PBL curriculum. 
The rest of the students, who were enrolled in the conventional curriculum, composed the 
second sample. Two standardized tests on Physics (the Force Concept Inventory and A 
Mechanics Baseline Test) were administered to all the students at the beginning of the first 
semester (pre-test) and at the end the fourth semester (post-test). Those tests were 
developed and reported by Hestenes, Wells, and Swackhamer (1992) and Hestenes, and 
Wells (1992) respectively. 



Mechanics Baseline Test 




Figure 1.- Comparison of PBL and control students on Mechanics Baseline Test’s scores 
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As can be seen in figure 1, ANCOVA analysis showed a significantly greater change in 
learning in PBL students in the Mechanics Baseline Test. In other words, although both 
groups’ scores improved from pretest to posttest, PBL students improvement was greater 
than control students’. Figure 2, on the other hand, showed that both groups (PBL and 
control) improved their scores in a similar way, with no differences in improvement 
between them. 



Force Concept Inventory 




Figure 2.- Comparison of PBL and control students on Force Concept Inventory’s’ scores 
Grade point average 

The second study compared the GPA obtained by students enrolled in the PBL curriculum 
with the GPA of a sample of control students enrolled in the traditional curriculum. 



Grade Point Average 




PBL Control 



t (77 J=4.552 

p<0.0001 



Figure 3.- Comparison of PBL and control students on grade point average means 
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This study also showed achievements that were significantly more favorable for PBL 
students As Can be seen in figure 3, when GPA means were compared for PBL and 
conventional students, PBL students significantly surmount conventional ones. 



Advanced engineering courses performance 



Finally a third study followed up students’ grades in advanced engineering courses. The 
purpose of this study was to test whether PBL built a stronger learning .transfer of basic 
sciences principles to applied fields. If this condition were true, it should be reflected by 
better achievement of the PBL students in specialized professional subjects, in comparison 

to the control students. 



In order to test this hypothesis, PBL and control students’ grades were compared for a 
sample of advanced courses that were selected by the program chairmen. 

As can be seen in figure 4, results showed that although PBL students were superior to 
control students in Mechanics, Electrical Circuits I, Digital Systems /, 1 erences e een 
the two groups were not significant. 



Mechanics 




P6L Control 



t (62p!.842 
p<0. 07013 



Electrical circuits I 




PBL Control 



t (48 pi . 8842 
p<0.0656 



Digital systems I 




PBL Control 



t (30 pi. 237 
p<0.2256 



Figure 4.- Comparison of PBL and control students on grade means in Mechanics, Electrical Circuits I and 

Digital Systems I 



However, statistical significance was achieved when PBL and traditional students grades 
in Probability and Statistics and Oral Communication were compared (see figure 5). Oral 
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communication was chosen because, even though it is not a topic related to engineering, it 
is very relevant to the PBL project: students work in teams every day, communicate to each 
other his/her ideas, and make oral presentations at least four times during each semester. So 
it was expected they would have a good performance in formal oral communication 
courses. It is also worth noting than in no case were traditional students superior to PBL 
students. 



Probability and Statistics Oral Communication 




t (44 >=3.907 p <0-002 

p<0. 0004 



Figure 5.- Comparison of PBL and control students on grade means in Probability and Statistics, and Oral 

communication 



Acquisition of skills and abilities 

One of the assumed benefits of problem based learning is the acquisition of generic skills 
by the students. As was explained before, our PBL curriculum is addressed toward the 
development of some skills, abilities and attitudes. So the second source of data used to 
evaluate the benefits of the PBL curriculum was related to the acquisition of professional 
skills. Three studies were carried out. The first compared students’ ratings of the degree in 
which the PBL/control courses contributed skill acquisition. The second study assessed 
inter-rater reliability of peer-evaluation and teacher-evaluation of skills and abilities. The 
third study compared PBL students and control students with regard to their performance 
on the California Critical Thinking Test. 

Students’ perceptions of PBL influence on the development of skills and abilities 

In the first study, students compared the PBL curriculum to the remaining courses with 
regard to the development of 15 skills. Students rated on a 10-point scale the degree to 
which the PBL curriculum contributed to the development of each skill as compared to the 
remaining courses, so that a rating of 10 meant “much more”, a rating of 5 meant “equal”, 
and a rating of 1 meant much less. The skills and abilities involved were Leadership, 
Analysis-Synthesis, Critical Thinking, Communication, Teamwork, Information Searching 
and Management, Entrepreneurship, Quality and Excellence, Creativity, Management of 
Computing and Telecommunication Technologies, Coping with Excessive Work Loads, 
Autonomous Learning, Problem Solving, Learning, and Motivation. 
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Two control groups were used for comparison. The first one (CGI) involved students 
taking equivalent traditional courses (physics, math, computing) which were taught by 
teachers who also teach in the PBL program. The second control group (CG2) involved 
students taking equivalent traditional courses that were taught by teachers who do not teach 
in the PBL program. 



Table 1 

Students’ perceptions of PBL influence on the development of skills and abilities 



Skill 


CG2 


CGI 


PBL 


F 


P< 


Leadership 


6.19 


6.22 


8.5 


16.66 


0.001 


Analysis-synthesis 


7.23 


7.33 


9.19 


12.66 


0.001 


Communication 


5.79 


6.03 


7.73 


7.48 


0.001 


Teamwork 


6.09 


7.02 


9.23 


22.23 


0.001 


Information Searching and Management 


6.24 


6.80 


8.88 


19.83 


0.001 


Entrepreneurship 


6.68 


6.88 


8.96 


14.65 


0.001 


Creativity 


7.13 


7.18 


8.92 


8.73 


0.001 


Computing and Telecommunication 


5.38 


6.83 


9.15 


29.66 


0.001 



Skill 


CGI 


CG2 


PBL 


F 


P< 


Critical thinking 


6.69 


7.35 


8.76 


14.37 


0.001 


Quality and Excellence 


6.82 


7.06 


8.44 


7.19 


0.001 


Coping with Excessive Work Loads 


6.68 


6.76 


8.65 


9.10 


0.001 


Autonomous Learning 


7.31 


7.41 


8.73 


6.80 


0.002 


Problem Solving 


7.00 


7.54 


8.73 


9.74 


0.001 


Learning 


6.73 


6.91 


9.23 


13.45 


0.001 


Motivation 


6.41 


6.59 


8.69 


10.41 


0.001 



Results are summarized in table 1 . Means are reported, from the lowest to highest, for each 
skill under each condition (CGI, CG2, and PBL). F and p values are also reported. The 
upper portion of the table shows those skills whose CG2 means were the lowest. The lower 
portion of the table shows those skills whose CGI means were the lowest. In every case 
PBL scores were the highest. Continuous lines under the means represent no between- 
group significant statistical difference, according to post-hoc analysis. Therefore, score 
means in leadership, analysis synthesis, communication, entrepreneurship, creativity, 
quality and excellence, coping with excessive work loadings, learning and motivation 
showed no differences between GC1 and GC2. 



Discontinuous lines represent significant differences. Hence, CGI scores on teamwork, 
information searching and management, and computing and telecommunication abilities 
surpassed significantly those of CG2, while CG2 overcame CGI scores with regard to 
critical thinking, quality and excellence, and problem solving. As can be seen in the table, 
PBL students’ scores surmount significantly both control groups in every skill. The fact 
that between GC1-GC2 differences were not always in the same direction provides 
evidence that the gains observed in PBL students were derived from the program itself 
more than from the teacher’s characteristics. In other words, whether the teacher teaches in 
the PBL program or not didn’t seem to contribute to the results. 
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Inter-rater reliability of peer and teacher evaluation of skills and abilities 

The second study involved peer and teacher evaluation of the same 15 skills. Its purpose 
was to assess the way in which different observers (evaluators) develops similar 
interpretations and expectancies about skills and abilities in the same process of evaluation. 
Inter-rater reliability on the judgments of skill acquisition was obtained. As it was pointed 
out. during the academic semester students worked in teams of four students. At the end of 
the semester, each student assessed their team partners on every skill so that average ratings 
were obtained for each student. Two teachers also rated every student independently on the 
same skills. In order to assess inter-rater reliability, correlation coefficients were obtained 
between teachers’ ratings and between each teacher’s ratings and the student average 
ratings. Between-teacher reliability was consistently high in each skill (average correlation 
coefficients of 0.81). These results are shown in the second column on table 2. 

Table 2 

Inter-rater reliability of peer and teacher evaluation of skills and abilities 



Skill/ability 


Teacher 1 - teacher 2 
reliability 


Teacher 1 - 
peer evaluation 
reliability 


Teacher 2 - peer 
evaluation 
reliability 


Autonomous learning 


0.84 


0.23 


0.39 


Coping with Excessive Work Loads 


0.86 


0.59 


0.67 


Analysis-synthesis 


0.83 


0.43 


0.41 


Information Searching and Management 


0.63 


0.32 


0,40 


Learning 


0.84 


0.59 


0,54 


Quality and Excellence 


0.83 


0.51 


0.62 


Communication 


0.79 


0.36 


0:43 


Creativity 


0.82 


0,35 


0,36 


Entrepreneurship 


0.70 


0.50 


0,56 


Computing and Telecommunication 


0.73 


0.31 


0.36 


Leadership 


0.91 


0.67 


0.60 


Motivation 


0.90 


0.41 


0.36 


Critical thinking 


0.79 


0.34 


0.59 


Problem Solving 


0.87 


0.69 


0.76 


Teamwork 


0.81 


0.38 


0.29 



When average peer ratings were correlated with teachers’ ratings, reliability was 
considerably lower (average of 0.46), although positive correlations were obtained in every 
skill, as can be observed in the third and fourth columns on table 2. In summary, teachers 
seem to develop a shared interpretation of the skills they are promoting and evaluating in 
their students. On the other hand, students’ judgments seem to be less “in tune” with 
teachers’ judgments. It is worth noting that when this study was conducted students were in 
their second academic semester. It could be expected that students will develop a more fine 
interpretation of the same skills in advanced semesters. 

Critical thinking evaluation 



The third study involved PBL and traditional students taking the Spanish version of the 
California Critical Thinking Skills Test, which assesses skills such as analysis, evaluation, 
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inference, inductive and deductive reasoning, as well as critical thinking skills which are a 
combination of the previous skills. 



Critical Thinking 




Figure 8.- Comparison of PBL and control students on critical thinking scores 

Figure 8 depicts students’ scores on each dimension. As can be seen, PBL students scored 
higher on each skill, as well as in the global score, than control students. However, as can 
be observed in table 3, between-group differences were non-significant. 

Table 3 

“t” and p values of the differences between control and PBL students on critical thinking 

scores 



Dimension 


Control 


PBL 


Student “t” 


p value 


Global 


13.07 


14.17 


1.3 


0.20 


Analysis 


3.07 


3.44 


0.96 


0.30 


Evaluation 


4.64 


4.83 


0.35 


0.73 


Inference 


5.35 


5.89 


1.09 


0.20 


Deductive 


7.00 


7.67 


1.09 


0.20 H 


Inductive 


4.62 


5.06 


0.83 


0.41 



Attitudes toward math, sciences and education 

The final set of data evaluated the attitudinal aspects of the PBL curriculum. PBL, it was 
hypothesized, should not only produce a better and more meaningful learning, but should 
also generate a more positive attitude toward the contents to be learned and toward 
education. Four attitude-rating scales were used for estimating students’ attitudes toward 
Computer Sciences, Physics, Mathematics, and Education. 
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Attitudes toward mathematics 

During the third academic semester, both control and PBL students were administered a 
Spanish version of the “Revised Math Attitude Scale” (Aiken, 1963) As can be seen in 
figure 9, PBL students’ attitudes toward math were superior, although non-significant, to 
control students’. It is worth noting, that one of the main problems that gave rise to the PBL 
curriculum was, precisely, the students’ reticence and perceived meaninglessness of 
mathematics and basic sciences. PBL seems to attenuate this negative attitude toward math. 



Attitudes toward Mathematics 




PBL Control p<0.07 



Figure 9.- PBL and control students’ attitudes toward Mathematics 
Attitudes toward education 

Similar results were obtained with regard to students attitudes toward education in general. 
The same sample of students answered “The Education Scale” (Rundquist, and Sletto, 
1967), and results are reported in figure 10. As in the case of mathematics, PBL students 
exhibited non-significant better attitudes toward education than control students. 
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Attitudes toward Education 




PBL Control p<0.16 



Figure 10.- PBL and control students’ attitudes toward Education 
Attitudes toward Computer Sciences 

The third field in which attitudes was explored was computer sciences. Students were 
administered an adaptation of the “Attitude Toward Computers” scale (Silance, and 
Remmers, 1934). As far as one of the program cornerstones was the use of information and 
telecommunication technologies, it was expected that PBL students would show a better 
attitude toward computer sciences. 



Attitudes toward Computer Sciences 




PBL Control p<0 05 



Figure 1 1- PBL and control students’ attitudes toward Computer Sciences 

Nevertheless, as can be observed in figure 11, PBL students’ attitude had a more negative 
trend than control students. Curricular changes are being implemented in order to make 
computer sciences teaching more meaningful. 
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Attitudes toward Physics 

Finally, Students answered the ’’Attitude Toward Physics Scale (Hand, 1953) which 
evaluates attitudes to Physics. As in the case of computer sciences, PBL students rated 
more negatively than controls. According to qualitative information, this fact seems to be 
attributable to teacher’s characteristics. However, a follow-up is being made with a 
different sample of students. 



Attitudes toward Physics 




PBL Control p<0.2 



Figure 12.- PBL and control students’ attitudes toward Physics 



Summary and future perspectives 

Although the PBL curriculum seems to improve the academic achievements in basic 
sciences learning by engineering students, some questions still need to be clarified. The 
main question that arises has to do with the students’ attitudes. Even though PBL students 
obtained remarkable outcomes in their academic performance, both during the four- 
semester (sophomore and freshmen) duration of the program, as well as during de higher 
semesters, their attitude toward the fields being learned did not seem to change accordingly, 
at least with respect to Physics and Computer Sciences. Some hypotheses may be 
speculated regarding the role of teacher’s attitude, very high requirements and pressure, and 
to curricular matters. However, additional studies need to be conducted in order to 
disentangle a satisfactory explanation. In fact, a pre-post attitude evaluation is being 
performed with a different sample of students. 

A second field that needs to be explored in more detail refers to skills and abilities 
acquisition. The data reported in previous sections of this paper suggests that PBL 
curriculum promotes some relevant skills. However all the evidence collected to date rests 
on student’s self-perceptions and teachers’ judgments. In order to get harder data on this 
matter, an Assessment Center technique is being planned, in which trained observers assess 



Effects of a PBL program on engineering students’ achievement, skills and attitudes 



17 



student’s evident performance under simulated conditions. This way, observers depart from 
a clear understanding of the skills being appraised, and the simulated conditions provide the 
circumstances that trigger the behaviors and interactions to be observed. 

Currently, a whole replication of the study is being conducted with some new generations 
of students, in order to assess changes and adjustments to the original program. Also, a 
follow-up is being planned for the first generation of students during their first years of 
their professional career. 

As a general conclusion it could be said that this project intended to extrapolate the PBL 
experience from Medical to Engineering Schools, as a vision of future that anticipates an 
answer to the needs of the current professional. At the academic side, it aims to produce a 
learning that expands the student’s action capability, a more enduring knowledge that does 
not find sense in information accumulation. Currently, the easiness in data searching leads 
us to emphasize critical thinking, and the ability to distinguish pertinent from irrelevant 
information. It is also worth noting that the work market often commits the professional to 
make changes in the job field. So it is necessary that he/she learn how to learn. In other 
words, that they develop skills that generate a more effective learning. 

In a world in which effectiveness and dignity are struggling against each other, it is 
important for human beings to leam to give meaning to their actions effectiveness, not as a 
solitaire hero’s achievement but because their impact in others. 

Finally, it is not possible to overlook the fact that the home of the future professional will 
be the virtual space. He/she will have to walk in the virtual world as he/she does at home. 
And a personal identity will emerge from that environment. For that to happen, the new 
citizens need to become acquainted and acquire the necessary skills that allow them to 
transform this new space in a place for their life-project realization. 
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